Add Inkling deployment helper for Akash#123
Conversation
There was a problem hiding this comment.
Pull request overview
What Problem This Solves
Inkling cannot be self-hosted on a single 24 GiB RTX 3090. This PR adds a helper that generates an Akash SDL for a small LiteLLM gateway that forwards requests to a remote OpenAI-compatible Inkling endpoint, making the non-self-host constraint explicit.
Why This Change Was Made
To preserve an Akash-based deployment option while steering users away from an impractical “single 3090” expectation and providing a concrete, reproducible gateway deployment path.
User Impact
Adds a new Bash deployment helper and documentation. Users running --dry-run / generating the SDL should be aware that secrets can be printed/written unless warnings/permissions are tightened.
Evidence
- Before: no Inkling-on-Akash helper script or dedicated setup doc in the repo.
- Tests: not provided in the PR metadata.
- Checks: not provided in the PR metadata.
- Autoreview: findings recorded on dry-run secret exposure, output-file permissions hardening, and a PR-description mismatch.
Changes:
- Adds
scripts/bash/deploy_inkling_akash.shto generate (and optionally submit) an Akash SDL for a LiteLLM gateway. - Adds
docs/inkling-akash.mdwith setup, cost/security notes, and a test example.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| scripts/bash/deploy_inkling_akash.sh | New helper script to generate an Akash SDL (CPU default, optional RTX3090 profile) and optionally deploy via provider-services. |
| docs/inkling-akash.md | New documentation for configuring, deploying, and testing the Akash LiteLLM gateway approach for Inkling. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if [[ "$DRY_RUN" == true ]]; then | ||
| printf '%s\n' "$SDL" | ||
| printf '\nDry run only; nothing was written or deployed.\n' >&2 | ||
| exit 0 | ||
| fi |
| umask 077 | ||
| printf '%s\n' "$SDL" >"$OUTPUT_FILE" | ||
| printf 'Generated %s\n' "$OUTPUT_FILE" | ||
| printf 'Security: this SDL contains secrets. Do not commit it; remove it after deployment.\n' >&2 |
| OUTPUT_FILE="deploy-inkling-akash.yaml" | ||
| PROFILE="cpu" | ||
| MAX_PRICE_UACT="25" | ||
| CPU_UNITS="0.5" |
| bash scripts/bash/deploy_inkling_akash.sh --output deploy-inkling-akash.yaml | ||
| ``` | ||
|
|
||
| Le fichier SDL contient les secrets. Ne le commitez pas et supprimez-le après le déploiement. |
What changed
scripts/bash/deploy_inkling_akash.shdocs/inkling-akash.mdwith setup, security, cost, and test instructionsWhy
Inkling cannot be self-hosted on a 24 GiB RTX 3090. The helper preserves the requested Akash GPU option while making the limitation explicit and providing a usable API-gateway deployment path.
Validation
--dry-runprints the generated SDL without writing or deploying--deployand anAKASH_KEY_NAMEImportant limitation
The RTX 3090 is not used for inference in API mode; a CPU-only Akash gateway would be cheaper. The documentation calls this out explicitly.